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Abstract 

Empirical evidence suggests that the Nash equihbrium leads to inaccurate predictions 
of human behaviour in a vast set of games. This observation has promoted the de- 
velopment of new sol u tion concepts like the quantal response equilibrium (QRE, see 



(Antal et al.. 


2009: 


Ohtsuki. 


2010) 



However, it is well-known 



that the QRE is subject to framing effects: Duplicating a strategy affects the equilibrium 
predictions. Here we show that the above mentioned evolutionary equilibria exhibit the 
same inconsistency. Furthermore, we prove that such framing effects are inevitable if a 
game theoretic solution concept depends differentiably on the payoffs. As a consequence, 
we argue that differentiable equilibrium notions, while being of great help in analyzing 
well-specified games, are unsuitable for theoretical modeling, where it is not clear which 
payoff matrix gives the true representation of an economic interaction. 
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1 Introduction 

For any class of games, static or dynamic, with complete or incomplete information, 
it is a simple task to create an exam ple where the Na s h equ ilibrium mispredicts human 
behaviour, as shown for example in iGoeree and Holtl ( 200ll ). This holds true even for 
the most simple games with only o ne rationalizable equilibrium, like in the traveller's 
dilemma introduced by iBasul (119941 ). Suppose that two travelers, returning home from 
their vacation, discover that the airline has lost their luggage. The airline asks both 
travelers independently to make claims for compensation and, in order to prevent excessive 
claims, determines that only the lower of both amounts will be paid. Additionally, it is 
announced that if the claims are different, the person with the lower claim obtains some 
reward R > 1, whereas the same amount R will be deducted from the other traveler's 
reimbursement as a penalty. In case that only claims between $180 and $300 are accepted, 
the Nash prediction is straightforward: In order to rake in the reward, it is always optimal 
to undercut the co-player's claim by one dollar. Consequently, the lower bound of $180 
is the unique equilibrium. While this analysis holds true for any R > 1, simple intuition 
suggests that subjects in the laboratory may try to coordinate on a higher claim if R is 
comparably low0 Indeed, this intuition is confirmed by experiments: For R = 5, around 
80 % of the subjects opt for the maximui n claim; only if i s suff iciently increased, claims 
approach the Nash equilibrium outcome (jGoeree and Holtl . l200ll ). Seemingly, subjects in 
these experiments do not strictly stick to best responses and do not necessarily eliminate 
dominated strategies. 

These observations are the starting point for several alternative equilibrium notions. In 
th is article we will review two d istinct examples, the quantal response equilibriu m (QRE) 
of 



McKelvev and PalfrevI (|l995l ) and the evolutionary equilibrium described in 



Ohtsuki 



(|2010l ). Instead of considering traditional steplike best response correspondences, these 
equilibrium notions assume that strategy choices are positively but imperfectly related 
to payoffsl§ As a consequence, also dominated strategies may be played from time to 
time, which in turn may affect equilibrium behaviour. Ironically, because these alternative 
equilibrium notions allow a more realistic description of human behaviour, they also have 



^As Kaushik Basu (1994) puts it, the strategy pair ("large" , "large" ) is a Nash equilibrium in ill- 
defined categories; if a player is told that the other player will choose a large number and if the reward 
R is neglectable, then the bes t reply is to choose a lar ge number as well. This explanation bears some 
similarity with the examples in lCamerer and Fehii (|2006l ). who describe under which conditions a minority 
of irrational agents can trigger a majority of rational individuals to mimic the minority's behaviour. 

•^The same idea has also b een applied to some learning models, for example smooth fictitious play, see 



Fudenberg and Levind (j 19981 ). 
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a serious drawback: These equilibrium notions themselves are subject to framing effects. 
Different representations of the same economic situation result in different predictions. In 
particular, giving a strategy a second alias may affect the position of the equilibrium. 

We proceed as follow s: In the next section, we review the QRE and the evolutionary 
equilibrium described in lOhtsukil ([20101). We show how two seemingly equivalent games 
can lead to diametrically opposed equilibrium predictions. While such framing effects 
are well-known in the case of the QRE, they have not been previously reported for the 
evolutionary equilibria. In Section [3] we give an unexpected sufficient condition for such 
framing effects: If an equilibrium concept depends differentiably on the payoffs then in- 
consistencies are inevitablelf] As a consequence it is argued in Section [5] that the QRE 
and other differentiable equilibrium notions, although being of great help in analyzing 
already specified strategic games, might be unsuitable for doing theory, where the true 
representation of an economic problem is far from being clear. 



2 Examples of equilibrium notions with framing effects 
2.1 The quantal response equilibrium 

The QRE was introduced by Richard D. McKelvey and Thomas R. Palfrey, f irst for games 



i n no rmal form (1995) and later also for extensive form games (1998). iGoeree et al 
(|2005l ) provide an axiomatic foundation. Since then, this concept was a pplied to various 
econom ic settings, including the traveler's dilemma (jCapra et al.l . Il999l ) or coordination 



games ([Anderson et al.l . l200ll ) . Typically, the QRE outplays the Nash equilibrium by far 
when it comes to predict human behaviour in laboratory experim ents]^ Remarkably, the 



QRE can also be used to estimate the rationality of the subjects (iMcKelvev and Palfreyl . 
1995 ) and to which extent they believe in their co-player's rationality ( Weizsackeil . 2003 ). 

For our purposes, it will be sufficient to consider the simplest finite normal 

form game between two players. The i?-player chooses a row of the matrix M = [a^i, bki), 
whereas the C-player chooses a column. As usual, players are allowed to randomize 
between their pure actions; we denote by and the respective mixed strategy vectors. 



Roughly speaking, diflerentiability means that smaU changes of the payoffs lead to a small and pre- 
dictable change of the equilibrium. Note that the Nash equilibrium concept does not satisfy this condition, 
since small changes in the payoffs may completely change the best respo nse correspondences. 

''The overwhelming success is illustrated by the following quote of [Camerer^ et al.[ ([2004[ '): Quantal 
response equilibrium, a statistical generalization of Nash, almost always explains the deviations from Nash 
and should replace Nash as the static benchmark to which other models are routinely compared. 
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For each player K € {R, C}, we denote by uf the expected payoff of K^s pure action i, 
which of course depends on the co-player's strategy , that is uf = uf{p~^). A main 
aspect of the QRE is that choice probabilities are positively but imperfectly related to 
payoffs. According to the most commonly used parametrization of the QRE, the logit 
rule, the probability to play action i is determined by the following stochastic reaction 
function a: 

pf='y{uf) = ^'^'-'^-\r ' (1) 



_ _ exp(A • -uf ) 

Y,- exp(A • uf) 



The sum in the denominator ensures that the probabilities sum up to one. The pa- 
rameter A can be interpreted as a measure of rationality: A = means that actions are 
chosen randomly from the set of possible alternatives, whereas for large A the choice is 
increasingly biased towards the strategy with the highest payoff. Note that as long as 
A < oo, even dominated strategies get a positive weight. For analyzing data, the param- 
eter A is typically estimated using the maximum likelihood method. A logit equilibrium 
is then defined as a fixed point of the map a: A pair of mixed strategies p = {p^,p'^) 
is an equilibrium if for both players K G {R,C} and all their strategies i the following 
condition holds: 

pf = a{uf{p-^)). (2) 

Such equilibria always exist but need not to be unique. As A goes to infinity, logit 
equilibria approach Nash equilibria. Furthermore, the graph of all fixed points p contains a 
unique branch, starting at the centroid of the strategy simplex for A = and converging to 
a unique Nash equilibrium as A approaches infinity, implying that the logit equilibrium can 
be applied to the problem of equilibrium selection. Since the stochastic reaction function 
a depends differentiably on the payoffs for A < oo, by the implicit function theorem the 
same holds true for each branch of the graph of the logit equilibrialfl 
Let us illustrate the logit equilibrium with an example taken from 



Goeree and Holt 



(|200ll ). Consider the following coordination game in which players receive $1.80 for co- 
ordinating on the high equilibrium and $0.90 if they coordinate on the low equilibrium. 
Additionally, the column-player has an outside option that guarantees a safe payoff of 
$0.40: 



^ As we will see in Section [3l the smooth dependence on the payoffs plays a key role. It is valid not 
only for the logit equilibrium bu t for the QRE in genera l , sinc e stoc hastic reaction funct ions are generally 
assumed to be differentiable, see iMcKelvev and PalfrevI (|l995l ) resp. I Goeree et al] (|2005h . 
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Figure 1: The unique braneli of the logit equihbrium for the coordination game with 
one, respectively with two outside options {x = 160). a) In the game with one outside 
option, the value of x is small enough to enable subjects to coordinate on the high 
equilibrium, b) However, if the problem is represented differently, splitting the safe 
option in two, players are predicted to coordinate on the low equilibrium. 
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This game has two pure Nash equilibria, {H, H) a ,nd (L, L), and the safe o ption is never 
part of an equilibrium. Nevertheless, as shown in iGoeree and Holtl (j2004l ). the outside 
option has a deciding influence on coordination behaviour in behavioural experiments. In 
particular, the exact value of x controls which strategies are chosen, with sufficiently low 
values of x prefering the (i?, H) equilibrium. Such an effect is correctly predicted by the 
logit equilibrium but not by the Nash equilibrium, see Fig. [1^ for an example with x = 160: 
The unique branch of the logit equilibrium, starting in the center for A = converges to 
the high equilibrium in the limit of rational agents, A — >■ oo. To illustrate that the QRE is 
subject to framing effects, we consider the same game, but with the second player having 
two (identical) outside options: 
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(3) 



While this additional strategy has no effect on the Nash equilibria, it alters the set 
of logit equilibria (Fig. [Da): Giving the outside option a second name leads the logit 
equilibrium to select the low instead of the high equilibrium. Thus, the prediction of the 
logit equilibrium depends sensitively on the exact formulation of the alternatively possible 
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strategiesjfl 

2.2 Evolutionary equilibria 



Ohtsukil ([20101) considers the following model of an evolutionary dynamics for an asym- 
metric game: There are two populations, a population R of row-players and a population 
C of column-players, with population size Nn resp. Nc- Each player in R chooses a row i 
of the matrix M = (a^i, b^i) € M™", whereas each player in C chooses a column j. Then, 
every subject in population R plays against every subject in the other population, leading 
to the payoffs and , respectively. Subsequently, the fitness fi of a player with strat- 
egy i is defined by an exponential transformation of its payoff, i.e. //^ = eyip{6uf) with 
K S {R, C}. The factor 5 measures the importance of the game for the fitness of a player 
and is usually called the strength of selection. If 5 — )• 0, each agent has approximately the 
same fitness, a case which is termed the weak selection limit. 

After those interactions, one subject (of any of the two populations) is chosen at 
random. This agent is allowed to change its strategy by imitating the strategy of another 
player of the same population. It is assumed that strategies with higher fitness are more 
likely to be adopted. More specifically, if denotes the current number of row-players 
with strategy k, then the probability that a randomly chosen row-player imitates an agent 
with strategy i is given by 

Additionally, one allows mutations: With probability u, the agent does not imitate others, 
but chooses randomly any of the available strategies. 

Overall, this evolutionary dynamics results in a stochastic selection-mutation process 



"^In the above example one might argue that the inconsistency can be avoided if identical columns 
are omitted by definition. However, if the game is marginally modified such that there are no identical 
columns, elimination of the additional column seems unjustified: 

& 

(4) 
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More fundamentally, it is typically not the subjects who construct payoff matrices to help them with their 
decisions, but it is the researcher who uses such tools to describe the decision maker's behaviour. How 
should one decide whi ch matrix gives the tru e repre sentation of the decision problem? Similarly, to adapt 
the argumentation of iKohlberg and Merteng (|l98(J ) on a related issue, elementary transformations, [like 
giving a column a second alias] , are irrelevant for correct decision making: after all, the transformed matrix 
is just a different representation of the same decision problem, and decision theory should not be misled 
by representation effects. To hold the opposite point of view is to admit that decision theory is useless in 
real-life applications, where problems present themselves without a special formalism. 
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without absorbing states. In some special cases, it is possible to explicitly calculate the 
invariant distribution of the process. One important case is the limit of weak selection, in 
which the fitness of each individual is largely independent of its payoff in the game. As 
a consequence, each strategy for population R is approximately played with probability 
1/m, only slightly truncated by a term ipi, which reflects the impact of the respective 
strategy. If both populations are of equal size this term is, up to a multiplicative constant, 
given by 

(fi = ai- a, (6) 

where Oj denotes the average of all feasible payoffs for a player with strategy i, that is 
= J2j o-ij/^j and a denotes the average of all feasible payoffs for individuals in population 
R, a = j (lij/inT-n)^ Since this mutation-selection process does not settle down on any 
stable state, the deviation terms (pi take the role of the major characteristic of the system. 
It is said that selection favors strategy i if y?i is positive (?). F urthermore, one may compare 



two different strategies with each other: lAntal et al.l (|2009l ) call strategy i more abundant 



than strategy k \i (pi > ipk. In effect, this approach allows a ranking of the strategies - 
based on the long run performance of each strategy in the above described evolutionary 
process. 

However, it is easy to show that this evolutionary equilibrium exhibits the same framing 
effects as the QRE. In fact, calculating (/? for the example in the previous section (for 
X = 60) yields tp^ = —5 and tp^ = 5 in the case of representation (I2D, respectively 
ip^ = 3.75 and ip^ = —3.75 in the case of representation Hence, neither does the 
absolute value of ip^ allow a consistent assessment across the different treatments, nor is 
the order of H and L left invariant. In the case of weak selection and uniform mutations, 
one can easily determine the reason for this inconistency: While in the first representation, 
the outside option is played by roughly 1/3 of all column-players, this fraction increases 
to approximately 1/2 of the C-population if there are two outside options, which in turn 
encourages row-players to choose strategy L. 



^This expression for ^pi resembles the well-known replicator dynamics, where it is assumed that the 
frequency of players with strategy i increases if the payoff Ui exceeds the average payoff in the population 
u (see, for example ?). However, while the payoffs m and u may vary over time, depending on the current 
state of population, the term ipi is constant and does only depend on the payoff matrix. 
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3 An Impossibility Result 

Let us turn to the question whether it is possible to construct other equihbrium notions 
(or other parametrizations of the QRE, respectively more general evolutionary equilibria) 
that avoid the inconsistencies shown in the previous section. For simplicity, we focus on 
two-player normal form games and identify each game T with its payoff matrix. For this 
reason, denote by = {{aij, hj) : Uij, bij G M} the set of all payoff matrices for normal 
form games, and let Alyt C be the set of all payoff matrices that have exactly k rows 
(i.e., that admit k pure strategies for the row-player). 

Definition 1 (Assessment formula) 

A game theoretic assessment is a function f : A4k — > K'^. 

One may interpret each entry of f{M) as the predicted equilibrium frequencies for the 
row-player in the normal form game defined by the payoff matrix In particular, note 
that the above definition of game theoretic assessments includes the logit equilibrium p 
and the evolutionary assessment ip from the previous section. 

In order to exclude framing effects, we demand that equivalent representations of a 
game lead to the same assessments. Up to a renumbering of the strategies of the column- 
player, we say that two matrices M, M £ Ai^ are equivalent if they result in the same 
matrix after deleting all columns that are a copy of a previous column. More formally, 
M ~ M if for all columns j of M there is a column / in M such that (ojj, bij) = {an, bn) 
for all rows i (and vice versa, for all columns / in M there is such a column j in M). 
Obviously, this defines an equivalence relation on the set M.k for all k. 

Definition 2 (Consistent assessment formulas) 

Fix a k > 2. An assessment formula f : A4k — > K*" is called consistent if it has the 
following properties: 

(i) Non-manipulability: IfM,Ke Mk and K ~ M then f{K) = /(M)§ 

(ii) Validity: If the row-player's strategy i is strictly dominated, then i cannot be optimal, 
/,(M) <max, /,(M). 

Non-manipulability means that a consistent assessment is well-defined with respect to 
the above equivalence relation, i.e. it respects that two matrices M, M with M ^ M 

*In this case one can restrict the image of the game theoretic assessment to the unit simplex A* instead 
of R . Since equilibria need not to be unique, this interpretation requires that for each game one particular 
equilibrium is selected out of the set of possible equilibria. 

similar condition can be found, for example, in Milnor's famous work on games against nature, see 
iMilnoj l|l95ll ). 
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represent the same game. Validity excludes constant assessment formulas from being 
considered. After these preparations, we are able to formulate the main result: 



Theorem 3 (An impossibility theorem) 

There is no assessment fromula that is both, consistent and differentiable. 

Proof. We show the case k = 2: Suppose there is such a consistent and differentiable 
assessment / and consider the arbitrary but fixed matrix 



M 



(an, 611) (ai2,6i2) 
(021,^21) (022,^22) 



Let df /doij denote the marginal change of the assessment if the row-player's payoff in 
the i-th row and the jth column is varied. The idea of the proof is as follows: Non- 
manipulability implies that all partial derivatives df/daij{M) are zero, which suggests 
that the value of f{M) is independent of the values of aij. This in turn contradicts 
validity. To show that the derivatives equal zero, we define the two matrices 



Mi{t) 



(aii+t, 611) (ai2,6i2) 
(021,^21) (022,^22) 



and 



M2{t) 



(aii+t, 611) (aii+t,6ii) (aii+t,6ii) (012,612) 
(021,621) (021,621) (021,621) (022,622) 

M2{t) is obtained from Mi{t) by doubling the first column two times. Note that Mi(0) = 
M. Next we define two functions that measure how the respective strategy assessments 
vary with t, i.e. we define Uj : R — )• M'^ with Ui{t) = fi^Miit)) for i = 1,2. Since 
Miit) ~ M2{t) for all t, non-manipulability implies that ui{t) = U2{t). In particular, 
the derivatives for t = coincide: 



Of 
dan 



(M) 



^M2(0)) + ^(M2(0)) + ^(M2(0)) = 4(0). 



dan 



12 



da 



13 



(7) 



Therefore, since we want to show df /daii{M) = 0, we have to compute the expression 
on the right hand's side of ([7]). For this reason, we define two new matrices: 



M3(t) 



(011,611) (011,611) (oii+t,6ii) (012,612) 
(021,621) (021,621) (021,621) (022,622) 



and 
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(an, 611) {au+t,bii) (aii+t,6ii) (012,612) 
(021,^21) (021,^21) («2i,^'2i) (022,^22) 

Note that these two matrices have the same reduced normal form and hence are equivalent. 
Additionally, they fulfill M^iO) = M4(0) = PhiO). If we again define functions Ui{t) = 
f(^Mi{t)) for i = 3,4, we may conclude that 



^3(0) 



df 
da 



13 



(M2(0)) 



'MM) + 



da 



12 



da 



13 



;m2(o)) = <(0), 



(8) 



and therefore df/dai2 (-/Vf2(0)) = 0. With a similar calculation one can show that the other 
two expressions on the right hand's side of ([7|) , 3//9aii (M2(0)) and 3//(9ai3 (M2(0)) , 
vanish as well. Therefore, we indeed end up with df /daii{M) = 0. A symmetry argument 
then immediately implies that df/daij{M) = for all i and j. As a consequence, the 
assessment f{M) does not vary in the row-player's payoffs, which leads to a contradiction 
with the validity of the assessment. □ 

Therefore, we must conclude that there is no reasonable equilibrium concept that is 
both, non-manipulable and smooth. If we interpret the value of fi{M) slightly differently, 
as an indicator of the performance of strategy i, then the previous theorem states that is 
impossible to measure the success of a strategy with a differentiable formula. 

In particular, the inconsistencies of the evolutionary assessment ip cannot be simply 
attributed to the assumption of weak selection. Eve n in the case of some positive but finite 
selection pressure 5, the stationary distribution in lOhtsukil (|20ld ) depends differentiably 
on the entries of the payoff matrix. 



4 Discussion 

Explaining human behaviour with game theoretic models faces at least two difficulties. 
Firstly, the modeller does usually not know the exact subjective utilities of the agents; 
instead there might be only some rough estimates. In order to obtain robust results, one 
might therefore require that the output of the model depends differentiably on the input 
data. Secondly, in order to set up the model, the researcher needs to choose one specific 
description of reality, out of many alternatively possible descriptions. One such choice 
might entail, for example, to determine whether a certain player has only one outside 
option or several similar options. In the best case - if the methods are consistent in the 
sense defined above - the exact representation of the game does not affect the qualitative 
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results. 

However, as we have shown, the two requirements of consistency and differentiabihty 
are incompatible. If the results of a game theoretic equilibrium notion depend differ- 
entiably on the payoffs, then these results also depend on the representation. In this 
sense, solution concepts for games in strategic form are necessarily imperfect. Therefore 
any game theoretic concept that can be applied to normal form games faces the choice 
whether it violates one requirement or the other. The matrix presented as Tab. [1] attempts 
to give an overview over some choices that were made. It classifies some popular game 
theoretic tools according to whether they violate the smooth dependence on payoffs condi- 
tion or the non-manipulability condition. Of course, such a list is notoriously incomplete 
and each cell of this matrix might contain several other elements - with the exception of 
the cell that corresponds to the differentiable and consistent concepts. 

A natural question is then to ask which of the two requirements is the more indis- 
pensable one. Differentiable equilibrium notions, and in particular the QRE, are quite 
successful in predicting human behaviour for normal form games - once it is known which 
representation of the game the subjects choose. In laboratory experiments this is certainly 
no problem, since it may be assumed that the players' internal model of the game is close 
to the instructions that are provided by the experimenter (in particular it is likely that 
all subjects have a similar internal representation). From a behavioural point of view, 
the framing effects presented in the previous sections even seem to be a desirable feature 
- after all it is well documented that humans are subject to framing effects as wellS 
Psychologically, it is not unreasonable to expect that a duplication of the outside option 
increases the number of L players in game ([3|). The outside options may act as a coordi- 
nation device: Because both o ptions point to the low equilibrium, this equilibrium may 
be interpreted as a focal point (jSchellingj . Il96d ). 

However, if it comes to explain human behaviour in the field it is not at all clear how 
individuals perceive their interactions, let alone that these perceptions are comparable 
across subjects. For theoretical modeling, the above described framing effects are unde- 
sirable (or even dangerous). If an equilibrium concept leads to predictions that depend on 
the representation of the game (which is chosen by the modeler himself), then the results 
will be somewhat arbitrary in the best case and manipulable in the worst. 

A possible solution to avoid framing effects in differentiable equilibrium notions is 
to consider the equivalence class of a game. That is, instead of calculating the logit 



For the related question whether subjects in dynamic games play differently i f confr onted with different 
game trees that represent formally equivalent games, see lMcKelvev and PalfrevI (|l998l ). 
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Differentiable concepts 



ft 
o 

a 
o 



n 
o 

o 



ft 

o 

o 
o 

CD 

o 
o 



Behavioural equilibrium notions 

QRE (M cKelvev a nd Palfrey, 19951) 

Level-fc reasoning model 

(Stahl and Wilson, 1995) 

Noisy introspection (Goeree and Hold . I2OO4I I 



Smooth learning processes ^BBW 

Exponential fictitious play 

(Fudenberg a nd Levine . 199^) 

Long run equilibria for evolutionary 
processes with uniform mutations and 
smooth selection 

Moran proc ess ijAntal et alj, l2009l: 

lOhtsukiboiOl ) 




Non-differentiable concepts 
' Nash equilibrium (' Nashl . Il950 h 

Refinements of the Nash equilibrium 

Perfect equilibrium (Selten, 1975) 
I Proper equilibrium (Mverson, 1978) 

Approaches that apply the Nash equilib- 
rium to transformed utilities 

Fairness model of Fehr and Sc hmidtJ l|l999l ') 

Learning processes for which Nash equilib^ 
ria are rest points 

I Fictitious play (jBrownl . 1 19511 ) 
Replicator dynamics 



ijTavlor and Jonkeil [197^ ) 

Best response 



dynamics 



i Gilboa and Matsui . .1991 ) 



Long run equilibria for evolutionary pro- 
cesses with uniform mutations and best- 
reply selection 

Moran process with strong selection 
l|Fudenberg et all lioo^ ) 



Table 1: A classification of game theoretic concepts 
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equilibrium p\{M) of a game M one may calculate the set of possible logit equilibria 
PA([Af]) for all games that are equivalent to M, 



[M] 



M e Mk \ M M 



}■ 



(9) 



However, in this case, the logit equilibrium loses its ability to select a unique Nash equi- 
librium in the limit of rational agents, A — )> oo. Instead, most of the Nash equilibria of 
a game M (including all strict Nash equilibria) are predictable by the unique branch of 
p\{M) - if only the game is appropriately framed. Th erefore, it seems to me that the 
solution concept of the Nash equilibrium is (almost, see 
good as it gets. 



Selten 



1975 



Mversonl . 1 19781 ) as 
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